1. Field of the Invention
The present invention relates to an image processing system, an image processing method and a computer readable information recording medium, and, in particular, to progression order code data decoding processing.
2. The Description the Related Art
Along with a spread of the Internet, information is disclosed in a homepage established in a WWW server (referred to as a web server, hereinafter), and a general user can search and obtain variable sorts of information therefrom. In such a system in which the web server discloses information on the Internet and clients view the information, the information disclosed by the web server can be viewed with the use of a web browser from the client.
In the web server, documents described with a HTML source are stored for establishing the home page. As a result of the client's web browser accessing it, the information is displayed on a display device of the client.
Further, the web browser of the client can obtain required information by use of a link included in the homepage displayed.
Further, as a protocol for fragmentarily accessing an image managed by the server, and displaying it on the client, IIP (Internet Imaging Protocol) exists. The IIP is an optimum protocol for an image data file format called FlashPix, and enables requesting a part of image data through the network.
As a transfer protocol for transferring a compressed image between the server and client, JPIP (JPEG2000 Interactive Protocol) for JPEG2000 compressed images exists. The JPIP is a protocol for fragmentarily accessing a JPEG2000 file and displaying it.
By using JPIP and JPEG2000, accessing a partial area of an image and selecting a resolution is made possible. Further, a change in image quality, switching between color and monochrome and setting of response data limiting a size can be easily made according to the client's request.
When, by using JPIP, JPEG2000 image data is fragmentarily transmitted, received fragmental code data should be cached in the client for the purpose of decoding the image data in the client. This is because, each scalability code data of JPEG2000 is different from scalability data lower by one level. According to JPIP, as a stream type for transmitting code data, two types, i.e., JPP-stream and JPT-stream exist.
The JPP-stream means a way in which precinct divided code is transmitted according to an order of a code stream. As shown in FIG. 24, in this way, the entire image is displayed from the beginning, and the image quality of the entire image improves as time elapses. In this case, at the beginning, although the image quality is low, a general view of the image can be recognized. Accordingly, this way is suitable for the purpose that the image quality of the entire image improves earlier, and the contents can be recognized earlier.
The JPT stream means a way in which tile divided code is transmitted in an order of the code stream. As shown in FIG. 25, tile images of top left are displayed first, then top right tile images are displayed, bottom left tile images and then, bottom right tile images are displayed. Thus, the entire image is finally displayed. In this case, decompression processing is performed in tile units. Accordingly, a memory amount required can be reduced since a common memory can be used for each tile decompression processing. Further, since each code data is transmitted successively in tile units, transmission efficiency is higher. This way is suitable for the purpose of printing, storing, or updating image data from a client to a server.
Related Arts of Code Data is Described Now.
(1) Code Data Progressive Control:
Japanese Laid-open Patent Application No. 2003-23544 discloses coding with switching a tile progressive order according to the tile's image attribute.
Japanese Laid-open Patent Application No. 2004-40674 discloses progressive display in which a resolution level of JPEG2000 code data is increased in sequence from an image resolution level currently displayed, at high speed.
(2) Code Data Access Control:
Japanese Laid-open Patent Application No. 11-205786 discloses FlashPix, which is a multiple resolution expression, and IIP, which is a protocol for accessing it.
Japanese Laid-open Patent Application No. 2003-23630 discloses that, in JPIP, a client has a cache, and requests a server for transmitting code not yet cached.
Japanese Laid-open Patent Application No. 2004-274758 discloses transforming JPIP precinct stream (JPP-stream) into a JPEG2000 stream.
(3) JPEG2000 Code Sequence Control:
Japanese Laid-open Patent Application No. 2003-169333 discloses a code sequence producing apparatus producing a new code sequence by use of deletion or sorting of a code sequence, and has a relation to a code sequence editing processing in JPEG2000 described later for embodiments of the present invention.
(4) Progression Order Change:
Generally speaking, to change a code data progression order and perform decoding reproduction of code data on the side of a client, a server selects packets corresponding to respective progressives from among a plurality of packets included in code data, sorts the selected packets according to the respective progressive orders, and re-transmits the code data of packet group corresponding to each progressive. Accordingly, time is required for re-transmission, and, thus, this way is not suitable for real-time processing.
In particular, when a plurality of receiving terminals designate progression orders for common code data on a server, and generate code data, complicated processing is required such that transmission is made for the plurality of receiving terminals. To perform this processing in a real-time manner is difficult.
On the other hand, when lossless code data is transmitted, a client can freely edit code data, change a progressive order and perform decoding reproduction. However, since a code amount increases, a transfer load increases, in this case.
Japanese Laid-open Patent Application No. 2003-244676 discloses that for a plurality for users who designate different quality images, and corresponding images of qualities having different progressive orders can be dispatched simultaneously.
Japanese Laid-open Patent Application No. 2003-169216 discloses that when a client transforms to a progression order suitable for a display requirement, ZLP is applied to code data, not stored in a memory, and JPEG2000 code data is generated.
(5) Editing on Code Level of Code Data Conforming to JPEG2000 Standard:
Japanese Laid-open Patent Applications Nos. 2003-16933 and 2004-088425 disclosed JPEG2000 code data or frame editing technology.
According to JPEG2000, code data having a multi-layer configuration can be produced. A layer corresponds to an image quality and a reproduction image quality can be selected as a result of the layer being selected. For example, when code data is made of three layers, a layer 1 is reproduced for the purpose of obtaining an image of an image quality 1, which is a lowest image quality. Layers 1 and 2 are reproduced for an image quality 2. Layers 1, 2 and 3 are reproduced for a best image quality image.
By using code data having a multi-layer structure, a coding part can easily select a layer of code data before decoding it.
However, Following Problems may Occur:
(1) Progressive Order According to Output Type Information:
In decoding for printer output, upon performing printing processing in sequence, code data should be preferably completely decoded part by part (in a raster order). When decoding is incomplete, even for a part, printing cannot be performed. Further, when printing output is performed, a memory capacity can be reduced and processing can be performed at high efficiency when data is received in the raster order and processing is performed thereon.
On the other hand, in decoding for a display purpose, it is preferable that decoding is made gradually throughout the entirety of a display area, and thus, the entire image concept can be recognized at an earlier stage.
Thus, when output is made for a display purpose, when progressive decoding for each area is applied instead of progressive decoding for each layer or for each resolution, the entire image concept cannot be known at an earlier stage.
(2) Code Data Access Based on Output Type Information:
In JPIP, in response to a request of a client, a server returns code data having a form of a complete image file, tile part stream (JPT-stream), or precinct stream (JPP-stream).
The JPT-stream is the way in which sub-image code data is returned as a set of tile parts defined by JPEG2000 part 1 standard. The JPP-stream is a way in which sub-image code data is returned as a set of precincts defined by JPEG2000 part 1 standard.
When code data, accessed by the JPT-stream manner in sequence is reproduced, complete decoding is performed in sequence for each tile obtained from tile dividing. When code data, accessed by the JPP-stream manner in sequence is reproduced, decoding can be performed gradually of each small area (precinct) unit for a wide zone including a plurality of tiles. That is, in reproduction in access of JPT-stream, an area which is reproduced at once is limited to a tile area of the image entirety. In contrast thereto, in JPP-stream access reproduction, reproduction at once on the image entirety, not limited to a tile area, is made possible.
Therefore, for printer output for which rater order sequential complete decoding is preferred, complete image data is accessed for each part (tile) in JPT-stream, and is reproduced. On the other hand, for display output for which decoding gradually throughout the entire display area is preferred, code data is accessed in fine units gradually throughout the image area entirety in JPP-stream. Even when display is made for each part, code data should be preferably accessed in JPP-stream within the area.
(3) Progressive Decoding:
When an image is displayed in layer progressive, a user can view in such a manner that a contour gradually becomes clearer. When this is applied to a document including characters, photographs and so forth in a mixed manner, the photographs cannot be recognized until reproduction is made approximately completely since a medium tone cannot be seen from a beginning stage. On the other hand, the characters can be recognized before reproduction is made completely since the characters have sharp edges.
Further, when an image is displayed in resolution progressive in which resolution increases gradually, a user can view in such a manner that a defocused state changes into a focused state gradually. When it is applied to a document in which characters and photographs are mixed, general contents of the photographs can be recognized before complete reproduction. However, the characters cannot be recognized until almost complete reproduction is made.
(4) Change in Progression Order After Code Data Production:
When a Motion JPEG2000 motion picture display system is applied, that is, when Motion JPEG2000 is based on, and motion picture data is dispatched to a plurality of clients (receiving terminals) from a single server, processing contents for a single hierarchical file data (code data) differs according to a quality (progressive) designated by the client (receiving terminal).
For example, a case may be assumed where, for a server, a first client (receiving terminal) designates space resolution progressive while a second client designates image quality progressive. In such a case, the server bears a large load for meeting the requirements from all the clients, and real-time processing may not be achieved.
This is because, in a motion picture dispatch system, only one type of quality is allocated for one frame, and the server should prepare different types of qualities for the respective ones of the plurality of receiving terminals.
(5) Re-Transfer of Packets Required Due to Change in Progressive Order:
Packets should be re-transferred when progression order is changed by a client.
According to JPEG2000, a plurality of progressive order code sequence configurations can be selected. Thus, even after producing code data, a client can edit the code data transmitted by a server, exchange progressive orders, or delete not required code sequences.
For example, when the client changes resolution level and outputs when the code data transmitted from the server is not one configured in resolution progressive, the client re-configures for a required code sequence after production of code data on code level, produces code data and reproduces it. This is a case where, after decoding reproduction is performed up to a layer of layer progressive code data, code data is re-configured after resolution is changed, and decoding reproduction is performed.
In this case, when code data is produced in which partial level code sequence is deleted from code data based on progression order, a required code sequence may be lost after the progression order is changed.
In such a case, a reproduced image quality may include a problem. The problem is solved as a result of the required packets being re-transferred. However, real-time processing cannot be achieved. In particular, for a case where real-time processing is performed for streaming type motion data, a serious problem may occur.
(6) Achieving Both Frequency Visual Characteristics and Resolution Visual Characteristics:
According to JPEG2000, layer progressive suitable mainly for frequency visual characteristics level and resolution progressive mainly for resolution visual characteristics level are prepared. Ordinarily, one is selected, and code data configured by code sequences guaranteeing a permissible level is generated. For example, when the resolution progressive is selected, code data configured by code sequences in a certain resolution is generated. However, the code data does not guarantee both the frequency visual characteristics and the resolution visual characteristics.